Linear and Order Statistics Combiners for Pattern Classification
نویسندگان
چکیده
Several researchers have experimentally shown that substantial improvements can be obtained in difficult pattern recognition problems by combining or integrating the outputs of multiple classifiers. This chapter provides an analytical framework to quantify the improvements in classification results due to combining. The results apply to both linear combiners and order statistics combiners. We first show that to a first order approximation, the error rate obtained over and above the Bayes error rate, is directly proportional to the variance of the actual decision boundaries around the Bayes optimum boundary. Combining classifiers in output space reduces this variance, and hence reduces the ”added” error. If N unbiased classifiers are combined by simple averaging, the added error rate can be reduced by a factor of N if the individual errors in approximating the decision boundaries are uncorrelated. Expressions are then derived for linear combiners which are biased or correlated, and the effect of output correlations on ensemble performance is quantified. For order statistics based non-linear combiners, we derive expressions that indicate how much the median, the maximum and in general the ith order statistic can improve classifier performance. The analysis presented here facilitates the understanding of the relationships among error rates, classifier boundary distributions, and combining in output space. Experimental results on several public domain data sets are provided to illustrate the benefits of combining and to support the analytical results.
منابع مشابه
Theoretical Foundations of Linear and Order Statistics Combiners for Neural Pattern Classifiers
Several researchers have experimentally shown that substantial improvements can be obtained in diicult pattern recognition problems by combining or integrating the outputs of multiple classiiers. This paper provides an analytical framework to quantify the improvements in classiication results due to combining. The results apply to both linear combiners and the order statistics combiners introdu...
متن کاملRobust Combining of Disparate Classi ers through Order Statistics
Integrating the outputs of multiple classiiers via combiners or meta-learners has led to substantial improvements in several diicult pattern recognition problems. In this article we investigate a family of combiners based on order statistics, for robust handling of situations where there are large discrepancies in performance of individual classiiers. Based on a mathematical modeling of how the...
متن کاملAnalysis of Linear and Order Statistics Combiners for Fusion of Imbalanced Classifiers
So far few theoretical works investigated the conditions under which specific fusion rules can work well, and a unifying framework for comparing rules of different complexity is clearly beyond the state of the art. A clear theoretical comparison is lacking even if one focuses on specific classes of combiners (e.g., linear combiners). In this paper, we theoretically compare simple and weighted a...
متن کاملClassiier Combining through Trimmed Means and Order Statistics
| Combining the outputs of multiple neural networks has led to substantial improvements in several dii-cult pattern recognition problems. In this article, we introduce and investigate robust combiners, a family of classiiers based on order statistics. We focus our study to the analysis of the decision boundaries, and how these boundaries are aaected by order statistics combiners. In particular,...
متن کاملFixed and Trained Combiners for Fusion of Imbalanced Pattern Classifiers
In the past decade, several rules for fusion of pattern classifiers’ outputs have been proposed. Although imbalanced classifiers, that is, classifiers exhibiting very different accuracy, are used in many practical applications (e.g., multimodal biometrics for personal identity verification), the conditions of classifiers’ imbalance under which a given rule can significantly outperform another o...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- CoRR
دوره cs.NE/9905012 شماره
صفحات -
تاریخ انتشار 1999